Information drives business. Companies today rely to an unprecedented extent on online, frequently accessed, constantly changing data to run their businesses. Unplanned events that inhibit the availability of this data can seriously damage business operations. Additionally, any permanent data loss, from natural disaster or any other source, will likely have serious negative consequences for the continued viability of a business. Therefore, when disaster strikes, companies must be prepared to eliminate or minimize data loss, and recover quickly with useable data.
Replication technology is primarily used for disaster recovery and data distribution. Continuous replication technology often includes RAID based replication schemes (e.g., disk mirroring, parity, or the like). Continuous replication maintains copies of the data as the data is being written to by applications. Periodic replication is another technique utilized to minimize data loss and improve the availability of data in which a point-in-time copy of data is replicated and stored at one or more remote sites or nodes. In the event of a site migration, failure of one or more physical disks storing data, or failure of a node or host data processing system associated with such a disk, the remote replicated data copy may be utilized. For both continuous replication and periodic replication, in addition to disaster recovery, the replicated data enables a number of other uses, such as, for example, data mining, reporting, testing, and the like. In this manner, the replicated data copy ensures data integrity and availability. Additionally, replication technology is frequently coupled with other high-availability techniques, such as clustering, to provide an extremely robust data storage solution.
Data storage required for applications such as file systems and databases are typically allocated from one or more storage devices that are maintained as a “volume.” The “volume” may serve as a logical interface used by an operating system to access data stored on one or more storage media using a single instance of a file system. Thus, a volume may act as an abstraction that essentially “hides” storage allocation and (optionally) data protection/redundancy from the application. An application can store its data on multiple volumes. The content of a volume is accessed using fixed sized data units called blocks.
In very large distributed computer systems the data is distributed in number of data servers. The clients directly write data to the data servers with minimal points of interaction with any metadata server. The data servers typically have multiple LUNs (logical unit numbers) which have their own reserved storage space. Each LUN can have a large number of partitions, with the objects contained in these partitions. The files of distributed computer system file system are composed of the objects from any of the data server LUN partitions. The objects from various data servers are combined to form RAID groups.
In a virtual environment such as Hyper-V, performance of the virtual disk is best when fixed VHD is used as compared to dynamic VHD. Fixed VHD allocates all blocks in the file system when the VHD is created. Therefore, for example, a 24 GB fixed VHD will create a file that is about 24 GB. This is different from dynamic VHD which, for example, only allocates enough storage to store data that is actually written to the VHD. A 24 GB dynamic VHD may only occupy a few MB when it's created. Fixed VHD is good for performance, but it presents a challenge for an optimized replication process, where file system information is used to reduce the amount of data to be copied when a new mirror is added to a volume.
Since fixed VHD allocates all blocks in advance, a conventional implementation of an optimized replication process cannot skip any block used by the VHD even though not all blocks are used by the VHD file. Conventional implementation of “optimized mirror/clone creation” can only skip white spaces not in use outside of the VHD files themselves and not within the file systems they host. Thus a typical volume hosting many such large VHD files cannot reap any performance benefits by simply using file system information on the volume. This factor causes replication processes performed in the conventional optimized manner to be excessively time consuming and overly resource intensive.